Field of the Invention
One or more embodiments of the invention are related to the fields of computer graphics and image enhancement. More particularly, but not by way of limitation, one or more embodiments of the invention enable a method of image compositing directly from ray tracing samples, a secure rendering system that generates ray tracing samples with obfuscated position data, a video security and ray tracing samples compression system, a non-rasterized image streaming system that uses ray tracing samples, and a method of modifying ray tracing samples after rendering and before rasterizing.
Description of the Related Art
Image production pipelines, such as those employed in visual effects or computer graphics projects, often include compositing operations that manipulate images or frames. Illustrative compositing operations may for example modify the colors, appearance, visibility, or locations of elements within an image, or they may combine elements from different sources into a single image or frame sequence. Often a rendering stage precedes compositing; rendering may for example transform a three-dimensional model of a scene into one or more images that may then be manipulated via compositing operations. Rendering often uses ray tracing techniques, which simulate the path of light rays between the eye of the viewer of an image and the objects in a scene. The ray tracing renderer generates a large number of simulated light rays, and generates a ray tracing sample for the intersection of each ray with the objects in the scene. A ray tracing sample may for example include data on the identity of the object that the light ray hits, the location of the intersection, and the color at the point of intersection.
In traditional image production pipelines known in the art, rendering via ray tracing is followed by rasterization, which transforms the ray tracing samples from the renderer into a grid of pixels. The rasterized image (represented as pixels) then provides the input to the compositing stage of the pipeline. Compositing operations known in the art therefore manipulate the pixels of images that have already been rasterized.
A drawback of this existing process is that a large amount of information generated by the renderer is discarded when the renderer's output is rasterized into pixels. Compositing operations are therefore less effective and less flexible than they could be if they directly manipulated the renderer's ray tracing output. Historically, rasterization prior to compositing has been used because storage capacity and processing power were insufficient to support composting directly from ray tracing samples. However, with current storage capacity and with the processing power available in graphics processing units (GPUs) and multicore CPUs, compositing directly from ray tracing samples becomes feasible. Compositing directly from ray tracing samples provides several benefits compared to compositing from rasterized images, since the ray tracing samples contain more information and are typically at a finer granularity.
Another challenge with existing rendering systems is establishing and enforcing security so that rendered images and videos cannot be pirated or stolen. For example, movie studios or other content producers often face the risk that the content they are rendering is pirated. Pirating may be done by employees, contractors, or service providers. In particular, the possibility of pirating of rendered images and videos may dissuade studios and producers from outsourcing rendering to service providers, since they expect that by keeping rendering in-house they can mitigate the pirating risks more effectively. This reluctance to outsource may lead to inefficiencies since studios and producers must all reproduce the large infrastructure (with hundreds or thousands of servers, for example) necessary to render complex scenes with modern graphics and effects. Existing rendering systems cannot provide assurances that rendered images and videos will not be pirated. Encrypting rendered images post-rendering may not be sufficient, for example, since the rendering system may still generate or store the unencrypted rendered images prior to the encryption step, making them subject to piracy. There is a need for a system that provides an inherently secure rendering solution, wherein the output of the secure rendering system cannot be used by pirates because rendered images are obfuscated as an integral part of the secure rendering process.
For at least the limitations described above there is a need for a method of modifying ray tracing samples after rendering and before rasterizing.